Coordinating Momenta for Cross-Silo Federated Learning

نویسندگان

چکیده

Communication efficiency is crucial for federated learning (FL). Conducting local training steps in clients to reduce the communication frequency between and server a common method address this issue. However, it leads client drift problem due non-i.i.d. data distributions different which severely deteriorates performance. In work, we propose new improve performance cross-silo FL via maintaining double momentum buffers. One buffer tracks model updating direction, other direction. Moreover, introduce novel fusion technique coordinate We also provide first theoretical convergence analysis involving both standard SGD. Extensive deep experimental results show better than FedAvg existing SGD variants.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Coordinating Policy for Federated Applications

At the start of its present term of office in 1997 the UK government published a planning document promising ubiquitous access to Electronic Health Records (EHRs) held within the National Health Service (NHS). If such access is to become a reality then it is essential to guarantee confidentiality, since otherwise the media and the privacy vigilantes will prevent deployment. Among the rights inc...

متن کامل

Federated Meta-Learning for Recommendation

Recommender systems have been widely studied from the machine learning perspective, where it is crucial to share information among users while preserving user privacy. In this work, we present a federated meta-learning framework for recommendation in which user information is shared at the level of algorithm, instead of model or data adopted in previous approaches. In this framework, user-speci...

متن کامل

Federated Multi-Task Learning

Federated learning poses new statistical and systems challenges in training machinelearning models over distributed networks of devices. In this work, we show thatmulti-task learning is naturally suited to handle the statistical challenges of thissetting, and propose a novel systems-aware optimization method, MOCHA, that isrobust to practical systems issues. Our method and theor...

متن کامل

Federated Learning: Strategies for Improving Communication Efficiency

Federated Learning is a machine learning setting where the goal is to train a highquality centralized model with training data distributed over a large number of clients each with unreliable and relatively slow network connections. We consider learning algorithms for this setting where on each round, each client independently computes an update to the current model based on its local data, and ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence

سال: 2022

ISSN: ['2159-5399', '2374-3468']

DOI: https://doi.org/10.1609/aaai.v36i8.20853